Skip to content

fix(gazelle): Skip indexing py_binary rules if a corresponding py_library rule contains the same srcs #2822

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

yushan26
Copy link

@yushan26 yushan26 commented Apr 24, 2025

This PR ensures that py_binary rules are not indexed into Gazelle's IndexMap when there is a corresponding py_library rule with the same srcs. When both py_library and py_binary targets share the same file in their srcs, Gazelle previously indexed both under the same import path. This led to ambiguity and resolution errors, as Gazelle found multiple rules for the same language (py).

To resolve this, the PR updates Gazelle to skip indexing py_binary rules, allowing py_library to be the only import being indexed.

Testing
An integration test was added where both a py_library and a py_binary rule include the same script.py in their srcs. The test verifies that Gazelle correctly resolves the import to the py_library target.

@arrdem
Copy link
Contributor

arrdem commented May 7, 2025

I don't think that just not indexing py_binary sources is a workable solution. A pattern we've seen before is that users will want to consider sources part of a py_binary and explicitly not want to manage what's perceived to be a duplicate py_library target, instead expecting other use sites to depend on that fileset via the py_binary target.

This has obvious downsides in that the entrypoint script gets materialized among other things, but it's an existing and working use-case.

I think the behavior in this case should simply be to prefer the library target in case both are an option, not ignoring binaries entirely.

@linzhp
Copy link
Contributor

linzhp commented May 8, 2025

A pattern we've seen before is that users will want to consider sources part of a py_binary and explicitly not want to manage what's perceived to be a duplicate

Are you talking about the scenarios when py_binary is generated but not py_library?

@yushan26 yushan26 force-pushed the index-py-library branch from cbeaaf2 to d37082d Compare May 8, 2025 19:51
@yushan26 yushan26 changed the title Skip indexing py_binary rules Skip indexing py_binary rules if a corresponding py_library rule contains the same srcs May 8, 2025
@yushan26
Copy link
Author

yushan26 commented May 8, 2025

Gotcha, I updated the logic to skip indexing the py_binary rule only if there is a corresponding py_library rule that also has the samesrcs populated. Let me know if this makes sense.

@yushan26 yushan26 changed the title Skip indexing py_binary rules if a corresponding py_library rule contains the same srcs fix(gazelle): Skip indexing py_binary rules if a corresponding py_library rule contains the same srcs May 27, 2025
@yushan26 yushan26 force-pushed the index-py-library branch from d37082d to 75c143d Compare May 27, 2025 20:39
@yushan26 yushan26 requested a review from rickeylev as a code owner May 27, 2025 20:39
@linzhp
Copy link
Contributor

linzhp commented May 27, 2025

@dougthor42 can you also take a look at this PR?

One thing I am not sure is whether Gazelle can generate py_binary without generating py_library like @arrdem said.

Copy link
Collaborator

@dougthor42 dougthor42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing I am not sure is whether Gazelle can generate py_binary without generating py_library.

In general, yes this is possible. Given

# foo.py
if __name__ == "__main__":
    print("hey")

Then, assuming gazelle:python_generation_mode file, Gazelle will generate:

py_binary(
    name = "foo",
    srcs = ["foo.py"],
)

However, package and project generation modes are different and TBH I'm less familiar with them.

@@ -86,6 +86,7 @@ END_UNRELEASED_TEMPLATE
multiple times.
* (tools/wheelmaker.py) Extras are now preserved in Requires-Dist metadata when using requires_file
to specify the requirements.
* (gazelle): Skip indexing py_binary rules if a corresponding py_library rule contains the same srcs: https://github.com/bazel-contrib/rules_python/pull/2822
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrap at ~80chars please

}
pyLibrarySrcs := otherRule.AttrStrings("srcs")
for _, src := range pyLibrarySrcs {
if src == pyBinarySrcs[0] {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A py_binary can have multiple srcs so you don't want to check against only the 1st value.

What are the requirements for not indexing the binary?

  • Any binary source file is found in any py_library srcs?
  • All binary srcs are found in a single py_library? Eg binary.srcs is a subset of library.srcs.
  • All binary srcs are found in multiple py_library targets?

@linzhp
Copy link
Contributor

linzhp commented May 31, 2025

Should we make Gazelle always generate py_library and stop indexing any py_binary? This is more consistent with other languages.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants